200 



206 



-216 

TRANSCODING 
ENGINE 



PATTERN REPLAY 



STATE 
RECOGNITION 




214 



205 



-^f USER 
-A DEVICE 





CONTENT 
RECOGNITION 
MODULE 



204 



210 



202 



INTERNET 



Fig. 2 



300 



302 



ACCESS A NETWORK SITE UTILIZING A NETWORK 



304 



SIMULATE USER INTERACTION FOR NAVIGATING THE NETWORK 
SITE BASED ON CLIENT SPECIFICATIONS 



306 



RETRIEVE A WEB PAGE FROM THE SITE 



308 



ASSIGN IDENTIFIERS TO ELEMENTS OF THE WEB PAGE 



310 



RENDER THE WEB PAGE FOR FURTHER PROCESSING UTILIZING 

THE IDENTIFIERS 



312 



RENDER DYNAMIC CONTENT OF THE WEB PAGE SUCH AS 
SCRIPTING UTILIZING THE IDENTIFIERS 



RETURN THE RENDERED WEB PAGE TO THE CLIENT 



314 



Fig. 3 




Fig. 4 



500 




Fig. 5 



600 



SEND REQUEST FOR URL TO USER 



SEND URL TO USER AGENT 



USER AGENT PARSES THE REQUEST 



RETRIEVE WEB PAGES ASSOCIATED WITH THE URL 



RECEIVE ACTIONS FROM THE USER 



602 



604 



606 



608 



610 




USER AGENT EXECUTES ACTIONS 



612 



614 



616 



RECORD THE ACTIONS 



SHOW RESULTING WEB PAGE 



618 



RECEIVE IDENTIFICATION OF PORTIONS OF WEB PAGE FROM 

USER 



620 



622 



RECORD THE IDENTIFIED PARTS 



Fig. 6 



n 












1 






pi, 0 








mi 




S^.; 








O 
O 




1^ 




CM 

d) 

■■■■ 

LL 



□ 

CI 

lU 
0 

m 

o 
m 
m 
m 
a 




d) 



1600 



1604 




Pattern Replay 
Engine (PRE) 



Iz. 



1602 



Session Manager 



1608 



Connector (e.g. 
XML Connector) 



Transcoding Page 
Rendering Engine 
(TRE) 



1610 



1606 



Fig 16 



1702 



RECEIVE A REQUEST TO TRANSITION FROM A CURRENT STATE 
IN A PATTERN TO A NEW STATE 



1704 



RETRIEVE INFORMATION ABOUT TRANSITIONING FROM THE 
CURRENT STATE TO THE NEW STATE FROM THE PATTERN 




1706 



EXECUTE AN ACTION USING THE SCRIPTING CODE FOR 
EFFECTING THE TRANSITION FROM THE CURRENT STATE TO THE 

NEW STATE 



1708 



1710 



RENDER THE NEW STATE FOR OUTPUT ON A USER DEVICE 



1712 



SENDING THE RENDERED OUTPUT TO A USER DEVICE 



Fig. 17 



RECEIVE A REQUEST TO TRANSITION FROM A CURRENT STATE 

IN A PATTERN TO A NEW STATE, WHEREIN THE STATES 
CORRESPOND TO CURRENT AND NEW STATES OF A REMOTE 

APPLICATION 



1802 



EXECUTE AN ACTION FOR EFFECTING THE TRANSITION FROM 
THE CURRENT STATE IN THE REMOTE APPLICATION TO THE NEW 
STATE IN THE REMOTE APPLICATION UTILIZING THE PATTERN 



1804 



1806 



GENERATING OUTPUT FROM THE NEW STATE 



1808 



SENDING THE OUTPUT TO A USER DEVICE 



Fig. 18 



1900 



1902 



HOST A WEB BROWSER 



1904 



RENDER A WEB PAGE OF A NETWORK SITE USING THE WEB 

BROWSER 



1906 



IDENTIFY A DYNAMIC ELEMENT WITH PRE-SPECIFIED 
PROPERTIES ON THE RENDERED WEB PAGE 



EXECUTE THE IDENTIFIED ELEMENT BASED ON A PRERECORDED 

PATTERN 



1908 



1910 



OUTPUT A WEB PAGE RESULTING FROM EXECUTION OF THE 

ELEMENT 



Fig. 19 



2000 



2002 



Connector (Interface) 



2004- 



2006 



HTML Connector 



SMTP Connector 



C3 

i 

a 
m 

e 
ai 
m 
m 
o 



2100 



2104 



Fig. 20 



State (Interface) 







Statelmpl 



2102 




Fig. 21 



2200 



2208 



2204- 



Pattern Replay 
Engine (PRE) 



SessionManager 



2206 



Scripting Engine 



Scripting API 



(via LiveConnect) 



Mozilla Rhino (3"^- 
party) 



-2202 



Fig. 22 



2300 



RETRIEVE CURRENT OUTPUT FROM A REMOTE APPLICATION 
UTILIZING A NETWORK, THE CURRENT OUTPUT CORRESPONDING 
TO A STATE OF THE REMOTE APPLICATION 



2302 



2304 



GENERATE AN OUTPUT STRUCTURE DESCRIPTION OF THE 
STATE OF THE REMOTE APPLICATION 



23.06 



COMPARE THE OUTPUT STRUCTURE DEFINITION TO A PRE- 
DEFINED STATE DEFINITION OF A RECORDED STATE 



2308 



DETERMINE WHETHER THE REMOTE STATE CORRESPONDS TO 

THE RECORDED STATE 



Fig. 23 



2400 



2402 



RECEIVE A PLURALITY OF DOCUMENTS 



ATTEMPT TO RECOGNIZE EACH OF THE DOCUMENTS AS BEING 
ASSOCIATED WITH A PREDETEREMINED STATE 



2404 



2406 



IDENTIFY A STATE BASED ON THE RECOGNITION OF THE 

DOCUMENTS 



Fig. 24 



2500 




Fig. 25 



User Input 
(action, etc.) 



2604' 



UAM 



Remote output 



2700 



2600 




2602 



XML 
Conn. 



PRE 



Fig. 26 




2504 



remote output 
vector of all HTTP 
states 



SRM 



2606 



StateData object 



remote output 



2702- 



IDT 



data elements' ids 
document/content 
description 




'state exists" (id) 



SRM 



2704 



state properties 



Fig. 27 



/ 

/- event 
-all 

connector 
state 

definitions 



-from PRE- 



Get request to 
execute an event 
or go to an initial 
state 



Send the 
Event to the 
User Agent 
Module 



2802 



2806 



2800 



2804 



Recieve Reponse 
from the User 

Agent 
(current output) 



Return error 
to PRE 



-Yes 




2608 



Send current output 
and state definitions 
to tlie SRIVI. 



■2610 



Yes 



Get new state id 
from the SRM 



Fig.28 




2612 



Return new state id 

and "content 
retrieval" object to 
the PRE. 




